Road surface management system and road surface management method thereof

ABSTRACT

According to one embodiment, a road surface management system inputs image data of road surface and capturing information of each of the image data and registers them. The image data are collected by repeatedly capturing a same route at predetermined distance. The capturing information includes information of a location and time at time of capturing. The system selects old and new images at a same spot from the registered image data, performs an association process, extracts an original image which is an old image at an arbitrary spot and a target image which is a new image associated with the original image unit based on the result of the association process, and outputs the original image and the target image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2019-193764, filed Oct. 24, 2019, theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a road surfacemanagement system and a road surface management method thereof.

BACKGROUND

In recent years, a camera comprising a global positioning system (GPS)function which can, every time an image is captured, obtain the locationinformation of the place has become widespread. Thus, an image to whichlocation information is added can be easily obtained. For example, in acase where a road surface is captured by a camera comprising a GPSfunction, an image data of the road surface with location informationcan be obtained. In a road surface management system, the following useis considered. For example, a road surface is captured at the same spotby a camera comprising a GPS function at regular intervals. The secularchange is observed by comparing the old and new images.

However, a camera comprising a GPS function has the following problemwhen it is used. Even if images at the same spot are to be extractedbased on the location information obtained by the GPS to compare old andnew images at the same spot, the images at the same spot cannot beextracted with high accuracy because of a measurement error of the GPS.

In a case where the coordinates of old and new images are integrated, amethod which enables the effective acquisition of corresponding pointsbetween the old and new images with high coincidence is suggested. Inthis method, a sufficient number of feature points are automaticallydetected from a plurality of frame images of a moving image. The featurepoints are automatically pursued between the frames. In this manner, CVvalue data indicating the camera position and rotation angle of themoving image is obtained. Thus, the coordinates of the images areintegrated. The feature of the method is as follows. Even in a casewhere the camera, the weather, the capturing condition or the directionof the observing point differs, corresponding points between old and newimages can be obtained without an effect caused by the differences. Theimages can be expressed in the same three-dimensional coordinate system.However, this method presupposes the automatic detection of a sufficientnumber of feature points.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the general structure of a roadsurface management system according to an embodiment.

FIG. 2A and FIG. 2B are diagrams showing an image data of road surfacecollecting vehicle used in the system shown in FIG. 1 and a group ofcaptured image data of the road surface.

FIG. 3A, FIG. 3B and FIG. 3C are conceptual diagrams showing thecontents of a process for extracting old and new images at the same spotby images in the system shown in FIG. 1.

FIG. 4 is a block diagram showing a functional block configurationaccording to the first implementation example of the system shown inFIG. 1.

FIG. 5 is a flowchart showing the flow of the entire process of thefirst implementation example shown in FIG. 4.

FIG. 6A and FIG. 6B are a flowchart showing the flow of a process forcalculating the feature amount of an original image and sample images inthe first implementation example shown in FIG. 4.

FIG. 7A and FIG. 7B are a flowchart showing the flow of a process forassociating images at the same spot with each other from the similarityof images and sample images in the first implementation example shown inFIG. 4.

FIG. 8 is a flowchart showing the flow of a process for associatingimages at the same spot with each other in a case where a plurality ofimages having the same similarity are present in the firstimplementation example shown in FIG. 4.

FIG. 9A and FIG. 9B are a conceptual diagram showing a processingexample in a case where a plurality of images having the same similarityare present in accordance with the processing flow shown in FIG. 8.

FIG. 10 is a diagram showing an example in which the similarity betweenan original image and a target image is calculated in the processingexample shown in FIG. 9A and FIG. 9B.

FIG. 11 is a conceptual diagram showing the result of association at thesame spot in the processing example shown in FIG. 9A and FIG. 9B.

FIG. 12 is a conceptual diagram showing how images are associated witheach other from the result of determination of a travel direction in thefirst implementation example shown in FIG. 4.

FIG. 13 is diagrams showing a screen display example displaying thelocations of the collected images and the images in the firstimplementation example shown in FIG. 4.

FIG. 14 is a block diagram showing a functional block configurationaccording to the second implementation example of the system shown inFIG. 1.

FIG. 15 is diagrams showing a screen display example in which the statusis unconfirmed in the second implementation example shown in FIG. 14.

FIG. 16 is diagrams showing a screen display example in which the statusis confirmed in the second implementation example shown in FIG. 14.

FIG. 17 is a flowchart showing another example regarding a method forrealizing the calculation of the similarity of old and new images in theimage association process shown in FIG. 5.

DETAILED DESCRIPTION

Embodiments will be described hereinafter with reference to theaccompanying drawings.

In general, according to one embodiment, a road surface managementsystem inputs image data of road surface and capturing information ofeach of the image data of the road surface and registers them in a firstregistration unit. The image data of the road surface are collected byrepeatedly capturing a same route at predetermined distance or timeintervals with an on-board camera mounted on a road traveling body andcomprising a function for obtaining the capturing information. Thecapturing information includes information of a location and time attime of capturing. The road surface management system selects old andnew images at a same spot from the registered image data of roadsurface, performs an association process, registers a result of theassociation process in a second registration unit, extracts an originalimage which is an old image at an arbitrary spot and a target imagewhich is a new image associated with the original image from the firstregistration unit based on the registered result of the associationprocess, and outputs the original image and the target image.

In the following explanation, the disclosure is merely an example, andproper changes in keeping with the spirit of the invention, which areeasily conceivable by a person of ordinary skill in the art, come withinthe scope of the invention as a matter of course. In addition, in somecases, in order to make the description clearer, the widths,thicknesses, shapes and the like, of the respective parts areillustrated schematically in the drawings, rather than as an accuraterepresentation of what is implemented. However, such schematicillustration is merely exemplary, and in no way restricts theinterpretation of the invention. In the specification and drawings, thesame elements as those described in connection with preceding drawingsmay be denoted by like reference numbers, and detailed descriptionthereof is omitted unless necessary.

EMBODIMENTS

FIG. 1 is a block diagram showing the general structure of a roadsurface management system according to an embodiment. FIG. 2A and FIG.2B are diagrams showing the image data of the road surface collectingvehicle used in the system shown in FIG. 1 and a group of captured imagedata of road surface. The road surface management system inputs a groupof image data of road surface collected by repeatedly capturing thesurface of the road of the same route at regular distances or timeintervals as shown in FIG. 2B with a mounted on-board camera comprisinga GPS function by a GPS receiver and an on-board camera mounted on avehicle (the target on which the on-board camera is mounted is notlimited to a vehicle and may be a traveling robot, and the mountedon-board camera may be a video camera having a GPS function) as shown inFIG. 2A, and also inputs the location of collection and the positionalinformation of the mounted on-board camera of each image. The roadsurface management system accumulates the input group of image data ofroad surface, location of collection and the positional information ofthe mounted on-board camera, searches for image data of road surface atthe same spot, performs an association process, outputs the result ofassociation of the same spot of the image data of the road surface, andin response to a request, extracts old and new image data of roadsurface at an arbitrary spot and displays the images such that theimages are compared with each other.

As an image capturing condition of the above mounted on-board camera,the road surface management system obtains a moving image or stillimages successively captured (for example, 15 images per second or atintervals of several meters). The mounted on-board camera faces thefront or rear and obliquely captures a road surface from the above(excluding the right above). The location information obtained by theGPS receiver is associated with an image.

FIG. 3A, FIG. 3B and FIG. 3C are conceptual diagrams showing thecontents of a process for extracting old and new images at the same spotby images in the system shown in FIG. 1. FIG. 3A shows a general methodfor associating an original (old) image with a target (new) image basedon the information of latitude and longitude of each image obtained byan on-board camera comprising a GPS function. Each of FIG. 3B and FIG.3C shows a method according to the present embodiment. FIG. 3B showsthat an original image having a great feature amount is specified from agroup of original images, and a group of target images having locationinformation which is substantially the same as the location informationof the specified original image is extracted. FIG. 3C shows that thefeature amount of each of the original image specified in FIG. 3B andthe images of the extracted group of target images is calculated, andthe original image and a target image having the same feature amount areassociated with each other.

In the general method for associating images based on the information oflatitude and longitude of each image, as shown in FIG. 3A, images ofdifferent places may be associated with each other because of an errorof the GPS. In this example, original image 3 showing pedestriancrossing 1 of original images is associated with target image 6deviating from image 4 showing pedestrian crossing 1 of target images.In the present embodiment, firstly, as shown in FIG. 3B, an originalimage (in this example, image 3 showing pedestrian crossing 1) having agreat feature amount is specified from a group of original images, andgroup 1-11 of target images having location information which issubstantially the same as the location information of the specifiedoriginal image 3 is extracted. Subsequently, as shown in FIG. 3C, thefeature amount of each of the specified original image 3 and the imagesof the extracted group 1-11 of target images is calculated. Originalimage 3 and target image 4 having the same feature amount are associatedwith each other. Since a group of original images and a group of targetimages are captured at regular distance intervals, old and new images atthe same spot can be extracted with high accuracy by carrying outassociation in the same arrangement as a plurality of association imagesbased on the feature amount.

First Implementation Example

FIG. 4 is a block diagram showing a functional block configurationaccording to the first implementation example of the system shown inFIG. 1.

The road surface management system of the first implementation exampleshown in FIG. 4 comprises an input unit 11, an input information storageunit 12, an image association unit 13, an image association resultstorage unit 14, a display control unit 15 and a display unit 16. Thissystem stores, in the input information storage unit 12, the image dataof each original image (old image) and each target image (new image)input by the input unit 11, and on-board camera location information(including the captured time and date, and the direction of the on-boardcamera). The image association unit 13 performs an association processfor images at the same spot or candidates for such images, using thelocation information of the old and new images stored in the inputinformation storage unit 12 and the features of the images. The imageassociation unit 13 stores the result of the association process in theresult storage unit 14. The display control unit 15 extracts a targetimage associated with an arbitrary original image based on the imagesand image association data stored in the input image storage unit 12 andthe image association result storage unit 14, and displays the result ofthe extraction on the screen of the display unit 16 such that the imagesare compared with each other.

FIG. 5 is a flowchart showing the flow of the entire process of thefirst implementation example shown in FIG. 4. In FIG. 5, when an inputimage is present in the input unit 11 (step S11), the input image isregistered in the input information storage unit 12 as an original image(step S12). Subsequently, when an input image is present in the inputunit 11 n (=1 to N) times (step S13), the input images are registered inthe input information storage unit 12 as target image n (step S14).After the registration of the images, in the image association unit 13,the feature amount of the original image is calculated, and an imagehaving the greatest feature amount is selected (step S15). Approximately10 images having the same latitude and longitude as the selected imagehaving the greatest feature amount are extracted from target image n (inthe case of intervals of 3 meters) (step S16). Subsequently, thesimilarity between the image having the greatest feature and each of theapproximately 10 extracted images is calculated (step S17).Subsequently, an image of target image n having the highest similarityis associated with the image having the greatest feature amount (stepS18). In the display control unit 15 and the display unit 16, theoriginal image and target image n are displayed on the screen, usingassociation information (step S19).

FIG. 6A and FIG. 6B show the flow of a process for calculating thefeature amount of the original image in the first implementation exampleshown in FIG. 4.

FIG. 6A is a flowchart. FIG. 6B shows a sample image in each processingstage. In FIG. 6A, when an original image is input (step S21),perspective projection conversion is applied (step S22, sample imageA1). The image is minified (step S23, sample image A2). The centralportion is cut out as the feature portion of the original image (stepS24, sample image A3). Subsequently, as shown in sample image A4 of FIG.6B, a template for searching for a range in which the number (amount) ofpixels of the transverse edge is the greatest is selected (step S25).Whether or not the number of pixels is greater than a threshold isdetermined (step S26). When the number of pixels is greater than thethreshold, the number of pixels of the feature portion is output as thefeature amount. When the number of pixels is not greater than thethreshold, the next original image is input, and the calculation processof the feature amount is performed. For a process for extracting anoriginal image having a great feature amount and calculating the featureamount, characterized images of a pedestrian crossing, a stop line, ajoint of a bridge, a manhole, etc., may be automatically extracted inadvance, using, for example, AI, and the feature amounts of the imagedata of the road surface may be calculated. An original image having thegreatest feature amount among the calculated feature amounts isextracted.

FIG. 7A and FIG. 7B show the flow of a process for associating images atthe same spot with each other from the similarity of the images in thefirst implementation example shown in FIG. 4. FIG. 7A is a flowchart.FIG. 7B shows sample images in a matching process. In FIG. 7A, whentarget images having location information which is substantially thesame as an original image (in consideration of image acquisitionintervals, approximately 10 target images) are input (step S31),perspective projection conversion is applied (step S32). The images areminified (step S33). The central portion of each image is cut out as thefeature portion of the target image (step S34). Here, as shown in FIG.7B, the similarity is calculated by comparing the feature portion of theoriginal image with the feature portion of the target image for matchingby normalized correlation (step S35). Whether or not the similarityexceeds a threshold is determined (step S36). When the similarity doesnot exceed the threshold, the next target image is input, and thefeature portion is cut out, and the similarity to the feature portion ofthe original image is calculated. When the similarity exceeds thethreshold, the target image is associated as an image at the same spotas the original image.

FIG. 8 is a flowchart showing the flow of a process for associatingimages at the same spot with each other in a case where a plurality ofimages having the same similarity are present in the firstimplementation example shown in FIG. 4. In FIG. 8, when an input imageis present in the input unit 11 (step S41), the input image isregistered in the input information storage unit 12 as an original image(step S42). Subsequently, when an input image is present in the inputunit 11 n (=1 to N) times (step S43), the input images are registered inthe input information storage unit 12 as target image n (step S44).After the registration of the images, in the image association unit 13,the feature amount of the original image is calculated, and a pluralityof images having a great feature amount are selected (step S45).Approximately 10 images having the same latitude and longitude as eachof the selected image data are extracted from a target image data n(step S46). Subsequently, the similarity between each image having agreat feature and each of the approximately 10 extracted images iscalculated, and association candidate images are selected (step S47).Subsequently, the distance between the images having a great featureamount is compared with the distance between the association candidateimages, and images having the same location relationship are associatedwith each other as images at the same spot (step S48). In the displaycontrol unit 15 and the display unit 16, the original image and targetimage n are displayed on the screen, using association information (stepS49).

FIG. 9A and FIG. 9B are a conceptual diagram showing a processingexample in a case where a plurality of images having the same similarityare present in accordance with the processing flow shown in FIG. 8. FIG.9A shows an original (old) image.

FIG. 9B shows a target (new) image. FIG. 10 is a diagram showing anexample in which the similarity between an original image and a targetimage is calculated in the processing example shown in FIG. 9A and FIG.9B. FIG. 11 is a conceptual diagram showing the result of association atthe same spot in the processing example shown in FIG. 9A and FIG. 9B.

Step 1: Firstly, an image having the greatest feature amount isextracted from the original image. Here, it is assumed that No. 6 isextracted.

Step 2: The similarity between No. 6 and target images is calculated(see FIG. 10).

Step 3: An image having a great feature amount (for example, greaterthan or equal to a threshold) is extracted from the original image.Here, it is assumed that No. 3 and No. 9 are extracted.

Step 4: Regarding No. 3 and No. 9, the similarity is calculated in thesame manner as step 2 (see FIG. 10). An image having the highestsimilarity is determined as the association target. In this way, theassociation target of original image No. 3 is determined as target imageNo. 4. The association target of original image No. 9 is determined astarget image No. 10.

Step 5: A target image corresponding to each original image isdetermined from the location relationships of original images No. 3, No.6 and No. 9 (see FIG. 11).

In this way, in the image association unit 13, when a plurality ofimages having the same similarity are present, the similarity iscalculated not only for an image having the greatest feature amount inthe original image but also for a plurality of images having a greatfeature amount, and an association process is applied, and the image ofthe association destination is specified from the location relationshipsof the images having a great feature amount (the distance between framesand the number of frames). This process also improves the accuracy ofthe association process.

FIG. 12 is a conceptual diagram showing how images are associated witheach other from the result of determination of a travel direction in thefirst implementation example shown in FIG. 4. FIG. 12 shows a functionfor automatically determining an approximate travel direction as asupplementary function of the image association unit 13. The traveldirection determination function of the image association unit 13 ofthis system performs a travel direction determination process from thecaptured date and time and location information as information added toeach image, adds travel direction information to each image, and storesthe information in the image association result storage unit 14. In thedetermination of the travel direction, the travel direction iscalculated from the difference in the information of latitude andlongitude of a plurality of images (successive, intervals of one second,etc.). For example, the travel direction is classified into eightdirections.

FIG. 13 is diagrams showing a screen display example displaying thelocations of the collected images and the images in the firstimplementation example shown in FIG. 4. In FIG. 13, (a) shows mapinformation indicating the locations at which the images are collected.In FIG. 13, (b) shows the images (original image and target images) atthe clicked location. This screen display example is displayed when auser operates the screen (in other words, when a user clicks theapplicable portion on the map). However, when location information isspecified, a search for the images of the applicable portion may beconducted, and the images may be displayed.

Second Implementation Example

FIG. 14 is a block diagram showing a functional block configurationaccording to the second implementation example of the system shown inFIG. 1. In FIG. 14, the same portions as FIG. 4 are denoted by the samereference numbers and explanations thereof are omitted. Here, differentportions are explained. The system shown in FIG. 14 comprises a feedbackfunction in a display control unit 15A and a display unit 16A. Thefeedback function of the display unit 16A is a function for, when a userselects and registers images at the same spot by operating the screen,registering the information in the storage unit 14 which stores theresult of association of old and new images. This function is used whenthe above image association process is not appropriately conducted orwhen a plurality of association candidates are extracted.

FIG. 15 is diagrams showing a screen display example in which the statusis unconfirmed in the second implementation example shown in FIG. 14. InFIG. 15, (a) shows map information indicating the locations at whichimages are collected. In FIG. 15, (b) shows the images (original imageand target images) at the clicked location. FIG. 16 is diagrams showinga screen display example in which the status is confirmed in the secondimplementation example shown in FIG. 14. In FIG. 16, (a) shows mapinformation indicating the locations at which images are collected. InFIG. 16, (b) shows the images (original image and target images) at theclicked location. When an image association process is not appropriatelyconducted or when a plurality of association candidates are extracted,the status is registered in the result of association as an unconfirmedstatus. As shown in FIG. 15, the original image at the locationdetermined as an unconfirmed status and target images which areassociation candidates are presented in series to encourage the user toselect and specify an image. When the user determines that the image isan image at the same location as the original image, the result ofassociation of the image in which the status is unconfirmed stored inthe image association result storage unit 14 is corrected (reflected)such that the target image is associated with the original image. Inthis manner, the above status is changed to a confirmed status. As shownin FIG. 16, the original image and the target image at the location canbe displayed as a confirmed status such that the images are comparedwith each other.

In the case of successive images captured at regular intervals(intervals of several meters), by one selection made by the user, aseries of images before and after the selection are automaticallyassociated.

FIG. 17 is a flowchart showing another example regarding a method forrealizing the calculation of the similarity of old and new images in theprocess for associating images with each other in FIG. 5. In FIG. 17,the same portions as FIG. 5 are denoted by the same reference numbers.Here, different portions are explained.

In FIG. 17, the image association processing unit adopts a method forcomparing images with each other regarding only a comparative roadsurface portion or the entire image including a background to calculatethe similarity while using deep learning for the calculation of thesimilarity of old and new images, and associating an old image with anew image based on the result of calculation. Specifically, the featureof the original image is learned (only a road surface portion or theentire image) (step S20). Using a dictionary (model) prepared bylearning, the feature of the original image is compared with the featureof target image n (step S21). The target image whose feature is the mostsimilar to the original image is associated with the original image(step S22). The process is transferred to the display control unit andthe display unit. Thus, even in a method for using deep learning for thecalculation of the similarity of old and new images, an image of targetimage n having the highest similarity can be associated with an imagehaving a great feature amount.

As described above, according to the road surface management system ofthe present implementation example, the user can easily confirm the samespot from the display contents of image data of road surface by thefeedback function and easily know the change or degradation in thesituation of the road surface by the comparative display.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. A road surface management system comprising: aninput unit which inputs image data of road surface and capturinginformation of each of the image data of the road surface, the imagedata of the road surface being collected by repeatedly capturing a sameroute at predetermined distance or time intervals with a camera mountedon a road traveling body and comprising a function for obtaining thecapturing information, the capturing information including informationof a location and time at time of capturing; a first registration unitwhich registers the image data of the road surface and the capturinginformation input from the input unit; an association processing unitwhich selects old and new images at a same spot from the image data ofthe road surface registered in the first registration unit and performsan association process; a second registration unit which registers aresult of the association process of the association processing unit;and an output unit which extracts an original image which is an oldimage at an arbitrary spot and a target image which is a new imageassociated with the original image from the first registration unitbased on the result of the association process registered in the secondregistration unit, and outputs the original image and the target image,wherein the association processing unit extracts, from the firstregistration unit, of the original image, an original image having agreat feature amount, and a group of target images having substantiallysame location information as location information of the original image,calculates a feature amount of each of the original image and images ofthe group of target images, associates the original image and a targetimage having a substantially same feature amount with each other, andassociates another original image with a target image in accordance witha result of association based on the feature amount.
 2. The road surfacemanagement system of claim 1, wherein the association processing unitcalculates the feature amount of the original image to select an imagehaving the greatest feature amount, extracts a plurality of targetimages having latitude and longitude substantially same as the selectedimage having the greatest feature amount, calculates similarity betweenthe selected original image and the extracted target images, andassociates a target image having the highest similarity with theselected original image.
 3. The road surface management system of claim1, wherein the association processing unit calculates the feature amountof the original image to select a plurality of images having a greatfeature amount, extracts a plurality of target images having latitudeand longitude substantially same as each of the selected original imageshaving a great feature amount, calculates similarity to the extractedtarget images with respect to the selected original images to selectassociation candidate images, compares a distance between the originalimages having a great feature amount with a distance between theassociation candidate target images, associates images having a samelocation relationship with each other as images at a same spot.
 4. Theroad surface management system of claim 1, wherein the associationprocessing unit applies perspective projection conversion to an inputimage, minifies the image of the result of the conversion, cuts out acentral portion of the minified image as a feature portion of theoriginal image, selects a template for searching for a range in whichthe number of pixels of a transverse edge is greatest, and outputs thenumber of pixels of the feature portion as the feature amount when thenumber of pixels based on the template is greater than a threshold. 5.The road surface management system of claim 2, wherein the associationprocessing unit calculates the similarity by comparing a feature portionof the original image with a feature portion of the target image formatching by normalized correlation.
 6. The road surface managementsystem of claim 1, wherein the association processing unit performs atravel direction determination process from the information of thelocation and time at the time of capturing included in the capturinginformation, and adds travel direction information to each image.
 7. Theroad surface management system of claim 1, wherein the output unitextracts a target image associated with an original image at anarbitrary location from the first registration unit, and displays aresult of the extraction on a display screen such that the images arecompared with each other.
 8. The road surface management system of claim7, further comprising: a feedback unit which reflects, when a userinstructs selection of an image at the same spot by a screen operationfrom the comparative display of the display screen, this information onregistered content of the second registration unit which registers aresult of association of old and new images.
 9. A road surfacemanagement method of a road surface image management system comprising:an input unit which inputs image data of road surface and capturinginformation of each of the images of the road surface, the image data ofthe road surface being collected by repeatedly capturing a same route atpredetermined distance or time intervals with a camera mounted on a roadtraveling body and comprising a function for obtaining the capturinginformation, the capturing information including information of alocation and time at time of capturing; a first registration unit whichregisters the image data of the road surface and the capturinginformation input from the input unit; an association processing unitwhich selects old and new images at a same spot from the image data ofthe road surface registered in the first registration unit and performsan association process; a second registration unit which registers aresult of the association process of the association processing unit;and an output unit which extracts an original image which is an oldimage at an arbitrary spot and a target image which is a new imageassociated with the original image from the first registration unitbased on the result of the association process registered in the secondregistration unit, and outputs the original image and the target image,the method being used for the road surface management system andcomprising: extracting, of the original image, an original image havinga great feature amount, and a group of target images havingsubstantially same location information as location information of theoriginal image; calculating a feature amount of each of the originalimage and images of the group of target images; associating the originalimage and a target image having a substantially same feature amount witheach other; and associating another original image with a target imagein accordance with a result of association based on the feature amount.